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ABSTRACT 

To determine whei-.her college students trained in the 
independent study strategy PORPE ("Predict, Organize, Rehearse, 
Practice, Evaluate"), could perform as well as other students trained 
to create and rehearse their own textually explicit and implicit tesu 
questions, a study compared performances on immediate and delayed 
multiple choice and essay exams, both dichotomously and holistically 
scored. Subjects ware 48 college freshmen enrolled in developmental 
readinq/study strategy courses at two state universities. Results 
indicax.ed that PORPE appears to offer content area instructors a 
comprehensive strategy which can help their students prepare for 
multiple choice and essay exams, a strategy which holds considerable 
promise for high risk students. These results generally corroborate 
those of the first study m demonstrating that PORPE is a 
comprehensive study strategy system which can facilitate student 
learning of psychology concepts, regardless of whether they are 
presented in recognition or recall formats. (One figure is included 
and 5 tables of data and 24 references are attached, as well as a 
master list of college reading and learning assistance technical 
reports.) (SR) 
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While experts continue to extol the integrated use of 
reading and writing as potent vehicles of learning, little 
evidence exists to show that students can independently use 
writing as a means of successfully mastering content area 
concepts (Langer, 1986; Langer & Applebee, 1987; Newell. 1984). 
The research that does exist focuses on three closely related 
approaches — summarization, analytical essays, and PORPE (Simpson, 
1986). Researchers have amassed considerable evidence tor using 
summarization as a means of remembering content area material 
when students have received training (Carr, Dewitz, Ogle, & 
Morningstar, 1987; Doctorow, Wittrock & Mark, 1978; Kings, Bigg, 
& Lipsky, 1984; Linden 8. Wittrock, 1981). Although this line of 
research seems initially encouraging, three concerns need to be 
acknowledged. First of aM , many studies have not been aDle to 
demonstrate decidedly that students trained in summarization will 
be able to score higher on objective ixams than students 
employing alternative strategies (King et al , 1984; Linden & 
V/ittrock, 1981). A second concern is that the task of writing a 
summary has always been defined by the researcher rather than the 
learner. Hidi and Anderson (1936) classify these summaries as 
reader-based. In contrast, wr i ter-based summaries are those that 
students produce for themselves, with no constraints on style, 
space, or structure, in order to facilitate and monitor their 
understanding and learning. A third concern has to do with an 
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overriding emphasis on easily and quickly quantified measures ot 
content (e.g., •number of idea units) to the exclusion of 
qualiltative measures that attempt to explain how a writer 
combines, synthesizes, and arranges separate pieces of 
information into a meaningful, whole discourse. 

Two other studies have investigated the analytical essays 
impact on student learning (Langer, 1984; Newell, 1984). Even 
though Langer's and Newel l^s research does address some of the 
limitations of the summarization studies, the effects of training 
students how to write the analytical essay are still unknown. A 
more serious limitation to the analytical essay studies and to 
the summarization studies is that they do not ..,^em to offer 
students the cognitive and metacogn 1 t i ve processes necessary to 
guide them through the processes of reading, studying, writing, 
and learning. 

A third line of research investigating the use of writing as 
a means of learning content has centered on PORPE (Predict, 
Organize. Rehearse, Practice, Evaluate). PORPE is an independent 
study strategy which operat ional izes the encoding and 
metacogni t ive processes that effective students engage in to 
understand and subsequently learn content area material (? mpson. 
1986). The steps, theoretical rationale, and research Dasis for 
PORPE are summarized in Figure 1. As stuaents independently 
employ the synergistic steps of PORPE they create 
learner-oriented essays which help them prepare for content 
examinations. Unlike previous summarization studies, students 
trained in PORPE have been able to significantly outperform a 
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similar control group on an immediate and delayed multiple choice 
exam over a psychology chapter excerpt (Simpson, Hayes, Stahl, 
Conner & Weaver, 1988)* Moreover, the essay answers (immediate 
and delayed) of the PORPE students were judged to be 
significantly superior when holistically scored for content, 
cohesion, and coherence. In this first study, however, the 
students trained in PORPE were compared to students trained in 
answering and rehearsing teacher-predicted questions. Vet to oe 
answered, however, is the question of whether or not PORPE would 
be as effective as another non-writing strategy more equivalent 
in the cognitive and metacogn i t i ve processes essential to 
1 ndependent 1 earn i ng. 



Insert Figure 1 about here 



Thus, this second study sought to determine whether college 
students trained in PORPE could perform as well as other students 
trained to create and rehearse their own textual ly explict and 
implicit test questions. Specifically, the research questions 
investigated were: 

1) Will students trained in PORPE perform significantly 
better on the combined multiple choice and essay exam (immediate 
and delayed)? 

2) Will students trained in PORPE perform significantly 
better on the immediule and delayed multiple choice exam? 
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3) Will students trained in PORPE perform significantly 
better on the Immediate and delayed essay exam dlchotomousl y 
scored? 

4) Will students trained In PORPE perform significantly 
better on the immediate and delayed essay exam holistically 
scored? 

METHOD 

Subjects 

The subjects were 48 college freshmen enroPed in 
developmental reading/study strategy courses at two state 
supported universities. The students were admitted to and 
enrolled in the respective universities through the developmental 
studies program because their Scholastic Aptitude Test scores 
and/or high school grade point averages were below basic criteria 
for regular admission to either university. They were not 
enrolled in any regular core college courses, but instead took 
only the required developmental reaaing, writing, and math 
courses. Two intact classes from each of the two universities 
were randomly assigned to each treatment conditio:, for a total 
of two experimental and two control groups. To insL'^e similarity 
across the four groups and across the two universities, the 
subjects'' scores on the Basic Skills Exam, a state mandatea 
reading test required of all entering college students, were 
collected and ....ed as a covariate in the data analyses. 
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Procedures 

For three weeks, the study involved five different phases 
which were Integrated into the normal classroom routine. In 
phase one the students in both groups were assigned to read three 
different psychology textbook excerpts^ the third one being the 
criterion passage for the data collection. All three excerpts 
were written on the college level and were between 1,658 and 
3,844 words long. During the second phase, training, the 
princir i researcher taught the subjects either how to apply 
PORPE or how to generate and answer textual ly exolicit and 
implicit short answer questions. In phase three, independent 
study, the psychology chapter excerpts were taken from the 
subjects who were then given two days to study their 
sel f-deve 1 oped training materials. During phase four, testing, 
the subjects were allowed 50 minutes to first answer the essay 
questions and then the multiple choice questions, both of which 
counted jn their course grades. A delayed and unannounced 
testing, phase five, occurred exactly two weeks later for the 
cr i ter i on passage , 

Data Source and Scoring Procedures 

The criterion exam contained 20 multiple choice items ana 
two essay questions, each worth 10 points, making a total of 40 
possible points. All three exams contained approximately 60% 
memory level questions and 40% higher level questions ceither 
interpretative or applicative). Piloted with similar students, 
the Kuder Richardson Formula 20 provided a reliability 
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coefficient of .79 for the criterion exam. Each exam also 
contained two essay questions which asked the students to 
discuss, compare, and contrast. 

The multiple choice exam was scored per question as having 
either a correct or incorrect response. The essays were 
independently and blindly scored by two raters In two different 
ways. The first scoring was done by two raters who used a 
dichotomous scale (Cooper & Ode 1 1 , 1977) with an interrater 
reliability of .87. This scoring procedure was meant to 
correspond to the procedure that a content area teacher generally 
employs when grading essay exams. 

The second scoring was a holistic assessment (Cooper, 1977) 
of three features for each essay: content, organization, and 
cohesion. Each feature was scored on a -i-^oint scale, with 4 
being the highest score. A rating of 0, however, was given on 
content If an essay failed to respond to the assignment; the 
scoring then ended for that essay with organization and cohesion 
also receiving a 0 rating. A rating of 0 was also g«ven on 
organization and cohesion if any essay merely listed information 
in phrases or unrelated, non-paragraphed sentences. The subjects 
received a subscore on each of the three features and a total 
holistic score, which was the sum of the subscores. Scores for 
each feature and a total holistic score were then summed (Myers, 
1980) across the two independent raters^ scores to produce a 
scale ranging from 0 to 8 for each feature, from 0 to 24 for an 
essays total holistic score, and from 0 to 48 for both essays. 
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A Pearson Product Moment correlation revealed an interrater 
reliability of .93 for the overall holistic scoring. 

RESULTS 

The analysis of covariance revealed no significant 
differences on the Basic Skills Exam across the four intact 
classes, F<2,46)= 1.14, £=.160. Each of the four research 
questions is discussed below, and adjusted means are available in 
Tables- 1 through 5. 

1) There were significant treatment effects for PORPE on the 
combined multipio choice and essay portions of the exam on the 
in-mediate [F<2 46)=17.67 £=.00013 and delayed [ F<2,46)=28.93, 
E=.0001] testings. 

2) While there was no significant treatment effect for PORPE 
on the initial multiple choice test [ P ':2,46)=2.22, £= .143], 
there was a significant treatment effect on the delayed multiple 
choice test t F(2,46)= 12.98, £=.001 J. 

3) There was a significant treatment effect for PORPE on the 
immediate C F(2,46)=27.55, £=.00013 and delayed 

[ F<2,46)=26.60 , £=.00013 di chotomousl y scored essays. 

4) There were significant treatment effects for PORPE on the 
immediate [ F(2,46)=9. 10 , £=.0043 and delayed [ F(2,46)=17.06, 
£=.00013 essays scored hoi i st i ca 1 1 y . Broken down for each of the 
three hoilstlcaliy scored features/ tral ts , the results were as 
foil ows : 

CONTENT: There were significant treatment effects on the 
content subscore for the immediate [ F(2, 46)= 10.613, 

Er|c 9 
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E= .002] and delayed [ FC2, 46)= 21.353, £=.0001] PORPE 
essays. 

ORGANIZATION: There were significant treatment effects 
on the organization subscore for the immediate 
I F(2,46)=13. 174, E= .0001] and delayed C F(2,46)= 
14.269, £= .0001] PORPE essays. 

COHESION: There were significant treatment effects on the 
cohesion subscore for the immediate C F(2,46)= 3.349, q= 
,07] and delayed [ F(2,46)= 13.988, £= .00013 PORPE 
essays . 

DISCUSSION 

The results of this second study generally corroborate the 
findings of our first study in demonstrating that PORPE is a 
comprehensive study strategy system which can facilitate student 
learning of psychology concepts, regardless of whether it be 
measured in recognition or recall formats. The single 
significant difference in findings between the two studies is 
that in this second study, the PORPE subjects did not score 
significantly higher on the initial multiple choice test. 
However, the potency and durability of PORPE were clearly 
demonstrated two weeks later on the delayed multiple choice test 
and even more clea-'y in the higher scores of the PORPE subjects 
on both the initial and delayed essay exams. 
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One plausible explanation for the lack of treatment effect 
between groups on the initial multiple choice test has to do with 
directed focus of attention and short-term memory. Because the 
control subjects were tau.ght how to construct study questions on 
the key ideas in the chapter excerpts, their attention was 
narrowly focused on the specific major points and supportive 
details that tnelr se 1 f -generated study questions emphasized as 
being important. This kind of attentive studying is useful in 
"cramming" for exams because it focuses the learner' s attention 
on a finite amount of information to be remembered, at least tor 
the short term. 

However, this narrowly focused approach of generating 
textual ly explicit and implicit short answer questions seems not 
be be as useful in preparing students for essay exams, which 
demand that students not only remember facts and major ideas, but 
also be able to articulate meaningful relationships among them. 
This micro-level study strategy, with Its emphasis on separate 
details or discrete chunks of informs, ion, seems not to encourage 
the learner to construct a coherent overview of the cor ?nt 
(Langer, 1986). Moreover, without an overworking and coherent 
schematic network of information to rely upon, the learner seems 
to lose the ability to recall information accurately and fully as 
time goes by. This loss of recognition and recall abi 1 i 1 ty may 
very well explain the lower scores of the control subjects on the 
delayed multiple choice test and on both essay tests. In other 
words, the learner-oriented essays that the PORPE subjects 
predicted, produced, and then evaluated may have required more 
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eiaborative processing and thus, greater depth ot processing 
(Bradshaw & Anderson » 1982). The answering of the sel t --predicted 
short-c\nswer questions, while initially a viable micro-level 
strategy, seems to lack the eiaborative processing and the 
integrative support system necessary for higher levels of 
thinking that extended writing can provide learners <Coe, 1987; 
Langer, 1986). 

The scores on the holistic analyses of organization and 
cohesion may offer some related clues to hf ]p explain why the 
PORPE subjects scored significantly higher than the control 
subjects on both the initial and delayed essay exams and on the 
delayed multiple choice exam. For instance, studies on writing 
quality have found that hiqher rated essays are also judged to be 
more coherent (Bamberg, 1983; Fahnestock, 1983; Witte & Faigley, 
1981). To give just one of many examples, Stotsky (1986) 
concluded that the "number and variety of interconnections among 
the semantic units" of a student's essay contribute to the 
essays clarity and overall quality. Moreover, she states that 
growth In the ability to create such meaningful and coherent 
texture reflects a commensurate growth in students* abililties to 
'^^ite about ideas. The essay exams that the control and the 
hJRPE subjects were asked to compose essentially measured their 
ability to understand, remember, and write about ideas. The 
superior scores of the PORPE subjects on content, organization, 
and cohesion strongly suggest that the steps of PORPE lad 
subjects to create more mearing^ful interconnections among ideas; 
certainly, they were better able to express meaningful 
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connections among ideas in their essay exams than were the 
control subjects. And It seems highly probably that tho^e 
interconnections allowed the subjects to remember the information 
longer and more accura ely than the control subjects. In short, 
the PORPE subjects'" superior scores on organization and cohesion 
support the claim that PORPE can contribute to deeper and more 
elaborative processing of content ideas. 

Finally, the results of this second study are similar to 
those reported by Marshall <1987) in a study investigating the 
roles that restrictive writing tasks (involving either no writing 
or written answers to short-answer questions) and extensive 
writing tasks (involving either personal analytic essays or 
formal analytic essays) play in secondary students' understanding 
of short stories. On both initial and delayed written posttests, 
students who had Initially responded to the short stories in 
extended writing scored significantly higher on measures of 
descriptive elaboration, interpretation, and generalization than 
did students who had initially responded to the stories in 
restrictive writing. The scores on the delayed posttests point 
even more strongly to the power of extended writing to improve 
the abilities of students to recall and interpret stories they 
had read days or weeks earlier. While Marshal rs study involved 
students' understanding of literary narrative, his findings are 
very similar to this study with expository text in that the PORPE 
trained subjects also demonstrated in their essays a superiority 
in learning over time. 
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IMPLICATIONS AND CONCLUSIONS 

PORPE appeals to offer content area instructors a 
comprehensive strategy which can help their students prepare for 
multiple choice and essay exams. This strategy can be initially 
introduced and taught by the instructor, but then gradually 
phased over to the students for their own control. Most 
importantly, PORPE is a strategy which holds considerable promise 
for high risk students, the subjects of this research study. If 
we listen to theorists and practitioners such as Berthoff ^J901), 
Dowst (1980), Elbow (1981), and FulvMler (1987), we learn the 
importance that writing plays in the creation of thought. Bu^ we 
must also remember than many high risk students have neither 
learned to appreciate the written word nor learned how to use 
langauge to construct or shape reality. PORPE provides an easily 
accessible set of integrated steps that can lead them to do so, 
for PORPE also teaches some of the equally necessary Intellectual 
processes that accompany the higher use of language — i.e., the 
related principles of focusing, selecting, organizing, composing, 
monitoring, and revising thought. In short, PORPE seems to 
operational ize the cognitive and me tacogn i t i ve processes that 
many high risk students need in order to succeed in college 
(Weinstein & Rogers, 1984). 
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Table 1 

Adjusted Group Means and Standard Deviations for the Combined Scores In the 
Multiple Choice and Dlchotomously Scored Essays (Initial and Delayed) 

Treatment M SD 
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Initial testing^ 25.25 4.43 

delayed testing^ 18.42 2.9? 



Questions-Answer 

initial testing^ 19.67 4*40 

delayed testing^ 13.21 2,56 



Maximum possible score = 40 



19 



Table 2 
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Adjusted Group Means and Standard Deviations for the Multiple Choice 
Test (Initial and Delayed) 



Treatment M SD 
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initial testing^ 14,63 2,10 

delayed testing^ 13,46 2,01 



Questions-Answer 

initial testing^ 13,79 3,26 

delayed testing^ 11,38 2,14 



Maximum possible score = 20 



2 0 
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Table 3 

Adjusted Group Means and Standard Deviations for the Dlchotomously 
Scored Essay Test (Initial and Delayed) 



Treatment M SD 
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initial testing 10.63 2,2 

delayed testing 4.96 1.1 



Ques t ions-Answer 

initial testing^ 5.88 3.62 

delayed testing 1.83 .56 



'Maximum possible score = 20 
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Adjusted Group Means and Standard Devic-itlons for Holistic Scoring of Iriclnl 
Essays 



Holistic C iteria 



Total Content Organization Cohesion 

Treatment M (3u) M (SD) M (SD) M (SD) 



PORPE 

29.29(2. 12)^^ 9.54(.72)'^ 10. 17 (.69)^ 10.00(.79)^ 

Question-Answer 

23.04(2.34)^ 6.54(.68)^ 7.50(.76)^ 9.0O(.95)^ 



^Maximum possible score = 48 



^Maximum possible score = 16 
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Table 5 

Acljusted Group Means and Standard Deviations for Holistic Scoring of Delayed 
Essays 

Holistic Criteria 

Total Content Organization Cohesion 

Treatment M (SD) M (SD) M (SD) M (SD) 

PORPE 

16.33(2.14)^ 5.00(.57)^ 5.5A(.77)^ 5.83(.93)^ 

Questions-Answer 

6.25(1.98)'' 1.58(.57)^ 2.25(.93)^ 2.A2(.99)^ 



Maximum possible score = 48 
Vlaximum possible score = 16 
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FIGURE 1 . Steps, theoretical constructs, and research basis for PORPE 



I 



TACTICS 


TACTICAL ACTION 


THEORETICAL CONSTRUCT 


RESEARCH BASIS 


PREDICT 


Predict possible essay questions on 
the material to clarify purposes 
for subsequent study. Identify 
critical aspects of text, and focus 
on major content . 


Encoding process of selection 
Processing text In an elaborativc 
manner 

Planning aspect of writing 


Cook & Kayer (1983) 

Redcr (1980), Rlckards & Divcsta (197«) 
Tlcrney & Pearson (1983) 


ORGAKIZE 


Organize key Ideas pertinent to the 
self-predicted essay question using 
one's ovn words, structure and 
oe thods • 

Summarize and synthesize Ideas via 
maps, charts, outlines. 


Encoding processes of selection, 
acquisition, and construction 

Planning and organizing Ideas for 
later vTltlng 


Cook & Kayer (1983), Wclnsteln & Mayer (1986) 
Tlcrney & Pearson (1983) 


REHEARSE 


Rehearse the organizational structure 
and key Ideas via active self- 
recitation. 


Monitoring, self-questioning, and 

taking corrective action 
Rehearsing proaotes learning 


Baker & Brown (198A) 

Anderson (1978), Gagne (1976), Smith (1967) 


PRACTICE 


Practice by vTlting an essay answer 
to the self-predicted question 
fron recall. 


Encoding processes of Integraticn 
Writing facilitates higher levels of 
thinking 

Writing requires a necessary fore to 

demonstrate understanding 
Writing is a generative process 


Cook & Mayer (1983) 

Emlg (1977), Langer (1986), Kewell (1986) 
Coe (1987), Richards (1936) 
Berthoff (1981), Stotsky (1986) 


EVALUATE 


Evaluate with a checklist the 
coDpleteness, accuracy, and 
appropriateness of the essay. 

A positive evaluation Indicates a 
readiness for the test. 

A negative evaluation indicates a 
need to loop back into the 
previous st eps of PORPE. 

2 1 


Self-regulating process of monitorinf,, 

checking, and evaluating 
Writing as feedback and rclnf orccaent 

to the 1 t^skim^T 


Baker & Brown (1986) 

Emlg (1977), Langer (1986) 
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